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ABSTRACT 


Many students are not good at geometric proofs. One reason for this is that it is possible to memorize the proof procedure 
explained by the teacher without understanding the structure of a geometric proof. So far, we have developed a 
web-based application that supports the understanding of the “structure of geometric proofs” in junior high school 
mathematics and have verified its effectiveness. Although the results of the evaluation experiments have indicated the 
usefulness of this web-based application, it was not possible to identify the factors that brought about its usefulness. In 
addition, because the web-based application was not equipped with a log analysis function, it was not possible to grasp 
the aspects that students struggle with during a proof in the web-based application. In this paper, we present the results 
and problems of the evaluation experiment of the web-based application so far and discuss the newly added log analysis 
function. 
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1. INTRODUCTION 


1.1 Challenges of Geometric Proofs in Junior High School Mathematics 


In eight grade mathematical geometry in Japan, according to the Ministry of Education, Culture, Sports, 
Science, and Technology (MEXT, 2008), the purpose of geometric proofs is “to understand the congruence 
of the figure, deepen the view of the figure, check the nature of the figure based on the congruence conditions 
of the triangle and logical considerations, and cultivate the ability to express the word”. To understand the 
needs and meaning of a proof which makes this logical rationality and deductive reasoning and its method, 
geometry can visualize the reasoning process (MEXT, 2008). In the second grade of junior high school, 
students are taught the assumptions and conclusions of geometric proofs at first. After understanding how to 
proceed with the proof based on the underlying evidence, students complete “fill-in-the-blank questions” and 
“short-answer questions”. A “fill-in-the-blank question” presents a completed proof with blank spaces that 
must be filled with the correct information. A “short-answer question” is a question where students must 
write proof sentences without using hints. In Japanese junior high school mathematics, learning instructions 
are being reported in order to try to understand geometric proofs using these two question types (Sekiguchi 
1994, Fujita 2014, Kurayama 2015). 

Looking at the national academic ability and learning situation survey over the past ten years conducted 
by the National Institute for Education Policy (2007-2016), the average correct answer rate for 
“fill-in-the-blank questions” is approximately 75%, while the average correct answer rate for “short-answer 
questions” is approximately 30%. The factors that cause this large difference in correct answer rate between 
the two question types are discussed with reference to the following two prior studies. 
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First, Moore (1994) classified the cognitive difficulties of proofs into seven categories, with one of them 
being “not understanding how to use definitions to capture the overall structure of a proof’. Next, according 
to an analysis by Koseki et al. (1987), among the students who could not prove the “short-answer questions”, 
approximately 40% understood that the three sides and corners of a triangle can be used to prove geometric 
joint. It was thought that although 40% of the students understood that the three sides and corners of a 
triangle can be used to prove a geometric joint that would lead to a correct answer, they were incorrect 
because they could not assemble them. In subsequent research, Kunimune et al. (2000) conducted the same 
survey for public secondary schools in an area close to the target area of the previous survey in order to 
clarify the situation after 12 years has passed. As a result, although a slight increase in comprehension was 
seen, approximately one quarter of students did not understand the mechanism of a geometric proof at the 
end of learning in junior high school. They continued to point out that it must be considered as a teaching 
issue. That is, based on the points of Moore (1994) and Koseki et al. (1987), the students’ understanding of 
the proof did not change over 12 years, and the students who could not answer the “short-answer question” 
correctly could not do so due to a lack of understanding of proof structure. 

An effective theory for this factor is the theory of multi-step formation of intellectual activity (Gal'perin 
2012). The multi-step formation theory of intellectual activity refers to idea that intellectual behavior is 
acquired through four stages: (1) explanation and presentation of intellectual activity, (2) activity using a 
specific object, (3) activity described in language, and (4) activity internalization. Based on these steps, this 
research developed and evaluated a learning support system that supports the structural understanding of 
geometric proofs. The structure visualization of the proof question is performed as the method of support. 

Based on this multi-step formation theory of intellectual activity, Hirashima (2015) reported the results of 
an “information structure-oriented approach to operating the logical structure itself’ for Japanese 
question-and-answer learning. Based on this, from the viewpoint of promoting the structural understanding of 
geometric proofs, it seems that the structuring of the learning content and its visualization are equally 
effective for different kinds of logical proof structures. However, although “fill-in-the-blank questions” used 
in conventional proof learning include the structure of the geometric proof, the student can complete the 
proof by focusing mainly on one sentence including blank spaces. Therefore, “fill-in-the-blank questions” are 
not sufficient for a proper understanding of geometric proof structure. Accordingly, proof learning that 
visualizes the structure of a geometric proof and includes a process that can actually be manipulated, 1.e., 
somewhere between the “fill-in-the-blank question” and “short-answer question”, is considered to be 
effective. 


1.2 Positioning of this Research 


Kurayama (2015) researched geometric proofs and said “If a student who answers a fill-in-the-blank question 
immediately answers a short-answer question, they must consider the whole from only a part.” Therefore, she 
has developed a system where students can use a card selection method to think about proof structure from its 
parts to learn it effectively. In her system, each element of a structured proof problem is provided as an 
“assumption card” and “condition card”. In this system, the student assembles while examining a simple 
sentence given as a card. It is thought that this system will be more focused on the “construction of proof” 
than the “structure of proof’. Further, Funaoi et al. (2009) developed a system where a proof is constructed in 
the form of a flow chart and errors are visualized after a correctness judgment. This system is designed to 
express “the flow of proofs” from the condition of the blank and to “recognize the error of the student” by 
visualizing it. To effectively learn a proof, it is important to consciously understand it through visualization 
of its structure and flow. It is difficult to understand the structure of a proof in terms of its components and 
what kind of relations they have. Therefore, it is considered that students can use the proof activity to 
imagine proof structure through a proof activity that applies proof pieces like a jigsaw. 

Based on the above, this research aims to design, develop, and evaluate a web-based application focusing 
on the structure of proofs, and an appropriate usage is suggested. This paper presents the results and 
problems of a previous evaluation experiment and discusses the newly added log analytics function. 
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2. OUTLINE OF THE WEB-BASED APPLICATION 


A web-based application was developed that enables the student to acquire an image of the “structure of a 
proof” while visualizing it and that embeds elements in each area of the hypothesis part. 

In this research, the development language was JavaScript. The structures of geometric proofs were set based 
on previous research, such as that by Kurayama (2015). The developed application can be implemented on 
both iOS and Windows devices. Figure | illustrates the configuration of the system. 

The web-based application has three steps for studying. The student-side user interface of step | is shown 
in Figure 2. The “proof structure board” that visualizes the “structure of geometric proof” on the left of the 
figure has an assumption part, intermediate joint proof part, and conclusion. Each area on this “proof 
structure board” is a figure-related piece arranged at the right side of the screen (e.g.,2AED=2CEB) or a 
figure property piece (e.g., the apex angle is equal); these pieces can be dragged around to assemble proofs. 
The right of Figure 2 is the screen after dragging the pieces. In addition, a “reason” is selected from the 
reason list in the joint proof section. If the piece cannot be placed, the piece will be returned to the first 
position. Once the proof has been assembled, the user receives a decision of correct or incorrect. If the proof 
was incorrect, the incorrect parts are highlighted as a basis for the student to answer again. 

The student-side user interface of step 2 is shown in Figure 3. In step 2, a function is provided for 
learning with consideration of the relationship between each element of the “proof structure”. In this step, the 
student constructs the “proof structure” themselves from five elements (assumption, reason, establishment 
items, triangle joint conditions, and conclusions) that constitute the proof, which are provided as pieces to the 
student. In this manner, from step | of acquiring an image of the whole proof structure, it becomes possible 
to be aware of the relation of each element, and it is predicted that the understanding of “proof structure” is 
promoted. The student-side user interface of step 3 is shown in Figure 4. In step 3, based on the “proof 
structure” created in steps 1 and 2, an activity of converting into “proof text” is performed. Specifically, 
“proof text” is provided as a piece for each word, such as “what if’, “because”, and “AB = CD”. Then, while 
referring to the “proof structure” presented on the left of the screen (see Figure 4), a “proof sentence” is 
assembled. 
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Figure 1. System configuration diagram 
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Figure 3. The student-side user interface of step 2 Figure 4. The student-side user interface of step 3 


3. EVALUATION OF THE WEB-BASED APPLICATION 


3.1 Results of Evaluation Experiment 


The evaluation experiment used the web-based application equipped with only step 1. The evaluation was 
performed with the cooperation of 33 junior high school students in the eighth grade. It was conducted 
according to three procedures: (1) pre-questionnaire and pre-test, (2) implementation of the task using the 
web-based application, and (3) post-questionnaire and post-test. In the pre-test, an awareness survey for 
simple proof questions and short-answer questions were given. In the implementation of the task using the 
application, we worked on the task while talking with multiple people around one tablet. Later, in the 
post-test, a “short-answer question” and an answer form on the “structure of proofs” were prepared at the 
same level as the pre-test. 

Five of the nine students who answered the proof question incorrectly in the pre-test answered the 
short-answer question correctly, and two answered correctly on the “structure of geometric proofs”. 
However, one student made a mistake in the writing part. 


3.2 Challenges for the Web-Based Application: Learning Analytics 


Based on the above, as a result of verifying this web-based application, it was found that the understanding of 
“the structure of proofs” was promoted. However, it can be understood that there are students who cannot 
write the “text of proofs” even if they can understand the “structure of proofs”, and it became apparent that 
this web application requires a further scaffolding function. Also, in the evaluation experiment, analysis was 
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performed using performance tests and questionnaires, but the data obtained from the web-based application 
cannot be taken into consideration. 

In this regard, one of the effective approaches is learning analytics. Learning analytics is defined as “the 
use of intellectual data, learner-generated data, and analytical models to discover information and social 
connections, and to predict and advise learning” (Siemens 2010). Learning analytics also contribute to 
improve learning environments, and examine learning effectiveness in students by collecting and analyzing 
data to assess learning behavior and performance (Pechenizkiy et al. 2014; Ogata et al, 2015). Yin et al 
(2019) investigate the successful learning patterns based on eBook reading logs. Shimada et al (2015) also 
investigated the successful learning pattern using eBook logs outside class. Learning analytics allow teachers 
to understand learning process that improve learning performance. Yamada et al (2018) developed and 
evaluated the effects of concept map system using eBook log, and indicated the successful learners tended to 
flip forward the pages and made a concept map that contains more nodes. Onoue et al (2019) developed 
concept map categorization system for teachers to improve instructional design. Thus, learning analytics is 
one of the most helpful perspectives for the improvement of learning environments. 

In this research, based on the concept of learning analytics, it is necessary to consider the data obtained 
from the web application and to conduct a detailed analysis in addition to the questionnaires and tests. This 
web-based application is intended to be used in classes. Therefore, there is a need for a function that enables 
the teacher to confirm what the student in question is finding difficult in the middle of a proof. Also, to 
consider appropriate usage patterns, including data during the use of the web-based application, analysis is 
also required. Therefore, in this paper, the log analysis function of the newly developed web-based 
application is considered. 


4. LOG ANALYTICS FUNCTION 


A function was added to visualize the log of the student in the application, as shown in Figure 5. This 
function accumulates the learning log in MongoDB in JSON format, and an interface was developed using 
JavaScript frameworks Vue.js and D3.js. On the left side of the screen, student IDs using this web-based 
application are arranged. On the right side of the screen, the operation history of each student is displayed in 
chronological order. A circle represents an operation of placing one piece. See Figure 6 for the location of 
each area described below. A red circle means placed in the assumed area, a blue circle means placed in the 
reason part of the joint certification area, a yellow circle means placed in the establishment area of the joint 
certification area, and a green circle means that a combination of conditions is selected from the three. The 
letters in these circles refer to the contents of the pieces placed in the area. Taking Figure 5 as an example, it 
can be seen that hamada placed the piece “AB = CE” in the assumed area at 17:52:47. 
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Figure 5. Log analysis function screen Figure 6. The location of each area 
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Such visualization of the operation history allows the teacher to confirm what difficulties the student had 
in the process of making a proof. For example, in Figure 5, because Takeda placed various pieces in the 
assumption and conclusion areas, it can be understood that the assumption and conclusion cannot be 
extracted from the problem sentence. Also, by visualizing the studnt’s operation history, it is possible to 
grasp how to proceed with the proof of the student who is good at the proof. Taking Figure 5 as an example 
again, Masuki and Hosoi clarify the relationship of figures that can be used from problem sentences and what 
should be shown to prove that they select the congruence conditions that can be used on them. It turns out 
that they can think of things paradoxically. Then, by displaying the behavior history of all students in a list, it 
becomes possible to compare the students with each other, making it easy for the teacher to grasp the 
characteristics of the students. In the future, in addition to the functions presented here, additional functions 
are planned to calculate and visualize the operation time and error rate for each question for all students. This 
will allow teachers to provide efficient instructions that focus on the problems faced as they teach their 
assigned classes. 


4.1 Log Accumulation Method 


The method of accumulating logs is described here. As shown in Figure 7, each time a piece is placed on the 
student’s screen, information is stored in a log array and sent to a database called MongoDB in JSON format. 
Here, a unique ID is assigned to each piece and area placed. Note that this web-based application has a 
function that allows the student to annotate figures displayed on the screen. This function is used with the 
library called fabric.js, and the drawing record can be converted to the json format by a command, so the 
drawing record is also saved in the JSON format. Therefore, each time a piece is placed, student ID, problem 
ID being worked on, area where piece is dropped, content of dropped piece, server time, device time, and 
drawing content are arranged in an array. The array is then saved in the database in JSON format. 
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Dropped_area:i, 
Dropped_piece:3, 
Server time:~, 
Device time:~, 

Drawing content:~ 
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Figure 7. Log accumulation method 
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4.2 Log Visualization Method 


Here, how logs are visualized is described. As shown in Figure 8, data is extracted for each student ID from 
the stored data, converted to a character string, and a graphic is created using the d3.js library. In addition, the 
color at the time of producing a figure was made to be the same as the color of each area, as shown in Figure 
6. It was predicted that the teacher would be able to grasp the order in which students consider proofs. 
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Figure 8. Log visualization method 


5. CONCLUSION AND FUTURE WORK 


In this paper, the evaluation results of a web-based application that supports the understanding of the 
“structure of geometric proofs” in junior high school mathematics were presented, and the newly added log 
analysis function was examined. In the future, we will evaluate the web-based application with the improved 
log analysis function and clarify the following two questions: 

“1. How did students use this web-based application effectively?” 

“2. What is the difference between students who are good and bad at proofs?” 

The following two points are proposed as proposals for solutions at this stage. For 1, it will compare the 
pre-test and post-test, and use the log analysis function to clarify the usage of students who have improved 
proof ability. As a result, it will continue to consider what kind of usage should be proposed in the future. For 
2 as well, the log analysis function is also used to refer to the history of students who are not good at proof 
and clarify where the problem is. And it is going to consider what kind of approach should be provided by 
the web-based application for the problem, and plan to provide a new function. Based on the above, the 
method of appropriate class use of this web-based application will also be discussed. 
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